Leader-Based versus Soft Control of Multi-Agent Swarms

We focus on the control of heterogeneous swarms of agents that evolve in a random environment. Control is achieved by introducing special agents: leader and infiltrated (shill) agents. A refined distinction is made between hidden and apparent controlling agents. For each case, we provide an analytically solvable example of swarm dynamics

Intent-based Deep Reinforcement Learning for Multi-agent Informative Path Planning

In multi-agent informative path planning (MAIPP), agents must collectively construct a global belief map of an underlying distribution of interest (e.g., gas concentration, light intensity, or pollution levels) over a given domain, based on measurements taken along their trajectory. They must frequently replan their path to balance the exploration of new areas with the exploitation of known high-interest areas, to maximize information gain within a predefined budget. Traditional approaches rely on reactive path planning conditioned on other agents' predicted future actions. However, as the belief is continuously updated, the predicted actions may not match the executed actions, introducing noise and reducing performance. We propose a decentralized, deep reinforcement learning (DRL) approach using an attention-based neural network, where agents optimize long-term individual and cooperative objectives by sharing their intent, represented as a distribution of medium-/long-term future positions obtained from their own policy. Intent sharing enables agents to learn to claim or avoid broader areas, while the use of attention mechanisms allows them to identify useful portions of imperfect predictions, maximizing cooperation even based on imperfect information. Our experiments compare the performance of our approach, its variants, and high-quality baselines across various MAIPP scenarios. We finally demonstrate the effectiveness of our approach under limited communication ranges, towards deployments under realistic communication constraints.Comment: \c{opyright} 20XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other work

Control of agent swarms in random environments

The collective dynamic behavior of large groups of interacting autonomous agents (swarms) have inspired much research in both fundamental and engineering sciences. It is now widely acknowledged that the intrinsic nonlinearities due to mutual interactions can generate highly collective spatio-temporal patterns. Moreover, the resulting self-organized behavior cannot be simply guessed by solely investigating the elementary dynamic rules of single individuals. With a view to apply swarm collective behaviors to engineering, it is mandatory to thoroughly understand and master the mechanism of emergence to ultimately address the basic question: What individual dynamics and what type of interactions generate a given stable collective spatio-temporal behavior ? The present doctoral work is a contribution to the general common effort devoted to give an engineering operational answer to this simple and yet still highly challenging question. Swarms modeling is based on the dynamic properties of multi-agents systems (MAS). Methodological approaches for studying MAS are i) mathematics, ii) numerical simulation and iii) experimental validation on physical systems. While in this work we strive to construct and analytically solve new classes of mathematical MAS models, we also make a very special effort to develop new MAS modeling platforms for which one is simultaneously able to offer exact analytical results, corroborate these via simulation and finally implement the resulting control mechanism on swarms of actual robots. In full generality, MAS are formed by mutually interacting autonomous agents evolving in random environments. The presence of noise sources will indeed be unavoidable in any actual implementation. This drives us to consider coupled sets of stochastic nonlinear differential equations as being the natural mathematical modeling framework. We first focus on the simplest situations involving homogeneous swarms. Here, for large homogeneous swarms, the mean-field approach (borrowed from statistical physics) can be used to analytically characterize the resulting spatio-temporal patterns from the individual agent dynamics. In this context, we propose a new modeling platform (the so-called mixed-canonical dynamics) for which we are able to fully bridge the gap between pure mathematics and actual robotic implementation. In a second approach, we then consider heterogeneous swarms realized either when one agent behaves either as a leader or a shill (i.e as an infiltrated agent), or when two different sub-swarms compose the whole MAS. Analytical results are generally very hard to find for heterogeneous swarms, since the mean-field approach cannot be used. In this context, we use recent results in rank-based Brownian motions to approach some heterogeneous MAS models. In particular, we are able to analytically study i) a case of soft control of the swarm by a shill agent, and ii) the mutual interactions between two different societies (i.e., sub-groups) of homogeneous agents. Finally, the same mathematical framework enables us to consider a class of MAS where agents mutually interact via their environment (stigmergic interactions). Here, we can once again simultaneously present analytical results, numerical simulations and to ultimately implement the controller on a swarm of robotic boats

Intearcting Brownian Swarms: Some Analytical Results

We consider the dynamics of swarms of scalar Brownian agents subject to local imitation mechanisms implemented using mutual rank-based interactions. For appropriate values of the underlying control parameters, the swarm propagates tightly and the distances separating successive agents are iid exponential random variables. Implicitly, the implementation of rank-based mutual interactions, requires that agents have infinite interaction ranges. Using the probabilistic size of the swarm’s support, we analytically estimate the critical interaction range below that flocked swarms cannot survive. In the second part of the paper, we consider the interactions between two flocked swarms of Brownian agents with finite interaction ranges. Both swarms travel with different barycentric velocities, and agents from both swarms indifferently interact with each other. For appropriate initial configurations, both swarms eventually collide (i.e., all agents interact). Depending on the values of the control parameters, one of the following patterns emerges after collision: (i) Both swarms remain essentially flocked, or (ii) the swarms become ultimately quasi-free and recover their nominal barycentric speeds. We derive a set of analytical flocking conditions based on the generalized rank-based Brownian motion. An extensive set of numerical simulations corroborates our analytical findings